Real-Time Synaesthetic Sonification of Traveling Landscapes

When travelling on a train, many people enjoy looking out of the window at the landscape passing by. We present an application that translates the perceived movement of the landscape and other occurring events such as passing trains into music. The continuously changing view outside the window is captured with a camera and translated into midi events that are replayed instantaneously. This allows for a reflection of the visual impression, adding a sound dimension to the visual experience and deepening the state of contemplation. The application can both be run on mobile phones (with built-in camera) and on laptops (with a connected Web-cam). We present and discuss different approaches to translate the video signal into midi events.